Image capturing apparatus and control method thereof

ABSTRACT

An image capturing apparatus includes an image sensor that has a plurality of two-dimensionally arrayed pixels, each of the pixels having a first photoelectric conversion portion and a second photoelectric conversion portion, a generation unit that generates a first image signal by connecting, in a pupil divided direction, a first signal obtained by combining signals of the first photoelectric conversion portions, and generate a second image signal, in the pupil divided direction, a second signal obtained by combining signals of the second photoelectric conversion portions. In a case of combing signals of the first photoelectric conversion portions, or combining signals of the second photoelectric conversion portions, the generation unit decreases weighting of a signal of a photoelectric conversion portion in which an effect of crosstalk from a neighboring photoelectric conversion portion is large.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a focus detection technique in an imagecapturing apparatus.

Description of the Related Art

Japanese Patent Laid-Open No. 2014-182360 discloses an apparatus thatperforms pupil division focus detection using an image sensor in which amicrolens is formed in each of two-dimensionally arranged pixels. Thisapparatus has a configuration in which one microlens is shared by twophotoelectric conversion portions. Accordingly, in a first photoelectricconversion portion out of the two photoelectric conversion portions thatshare the microlens, a signal that is based on a light beam passingthrough a first region in the exit pupil of a taking lens is obtained.In a second photoelectric conversion portion, a signal that is based ona light beam passing through a second region in the exit pupil of thetaking lens is obtained. By calculating the correlation between asequence of signals obtained from a plurality of first photoelectricconversion portions and a sequence of signals obtained from a pluralityof second photoelectric conversion portions, the phase difference(deviation amount) between the sequences of signals is calculated, and adefocus amount can be calculated from the phase difference.

In addition, it is possible to obtain an output similar to that of ageneral pixel having one photoelectric conversion portion for onemicrolens, by adding outputs of the first photoelectric conversionportion and the second photoelectric conversion portion that share themicrolens. Therefore, it is possible to obtain, from one pixel, threetypes of outputs, namely an output (an A signal) of the firstphotoelectric conversion portion, an output (a B signal) of the secondphotoelectric conversion portion, and an addition output (an A+B signal)of the first photoelectric conversion portion and the secondphotoelectric conversion portion. In Japanese Patent Laid-Open No.2014-182360, an A+B signal is read out after an output (e.g., an Asignal) of one photoelectric conversion portion is read out, and a Bsignal is generated by subtracting the A signal from the A+B signalwithout being read out separately. Accordingly, three types of signalscan be acquired by performing a readout operation twice.

In addition, Japanese Patent Laid-Open No. 2009-122524 disclosesexecution of focus detection after excluding, from outputs ofphotoelectric conversion portions, the effect of crosstalk ofneighboring photoelectric conversion portions in order to prevent adecrease in focus detection accuracy. Accordingly, it is possible toreduce the effect of crosstalk, and perform accurate focus detection.

However, in the case of performing focus detection after signalcorrection through crosstalk correction described in Japanese PatentLaid-Open No. 2009-122524 is complete, there is the following issue. Asdescribed in Japanese Patent Laid-Open No. 2009-122524, the amount ofcrosstalk that occurs changes according to not only an amount of pixeloutput that causes crosstalk, but also angle of incident light,F-number, image height of a focus detection region, area of aphotoelectric conversion portion, distance, and the like. Therefore, alarge number of pieces of accurate information is required in order toaccurately perform crosstalk correction, and it is difficult toaccurately obtain these pieces of information considering manufacturingerrors and the like.

On the other hand, Japanese Patent Laid-Open No. 2009-122524 does notmention the reliability of focus detection in the case where a certainamount of error remains even if crosstalk correction is performed. As aresult, a method for obtaining a reliable focus detection result interms of detection accuracy, in focus detection that is based on theassumption of an error remaining in crosstalk correction, is notdisclosed.

SUMMARY OF THE INVENTION

The present invention has been made in light of such an issue of aconventional technique, and provides an image capturing apparatus thatcan obtain an accurate focus detection result even in the case wheresignals include an error due to the effect of crosstalk, crosstalkcorrection, and the like.

According to a first aspect of the present invention, there is providedan image capturing apparatus comprising: an image sensor that has aplurality of two-dimensionally arrayed pixels, each of the pixels havinga first photoelectric conversion portion that receives a light beampassing through a first pupil region of an exit pupil of an imagingoptical system and a second photoelectric conversion portion thatreceives a light beam passing through a second pupil region of the exitpupil of the imaging optical system different from the first pupilregion; a generation unit configured to generate a first image signal,in a pupil divided direction, based on a first signal obtained bycombining a signal of the first photoelectric conversion portion to asignal of another neighboring first photoelectric conversion portion,and generate a second image signal, in the pupil divided direction,based on a second signal obtained by combining a signal of the secondphotoelectric conversion portion to a signal of another neighboringsecond photoelectric conversion portion; and a focus detection unitconfigured to detect a phase difference between the first image signaland the second image signal, wherein, in a case of combining a signal ofthe first photoelectric conversion portion to a signal of anotherneighboring first photoelectric conversion portion, or combining asignal of the second photoelectric conversion portion to a signal ofanother neighboring second photoelectric conversion portion, thegeneration unit decreases weighting of a signal of a photoelectricconversion portion in which an effect of crosstalk from a neighboringphotoelectric conversion portion is large, and performs combining.

According to a second aspect of the present invention, there is provideda controlling method of an image capturing apparatus including an imagesensor that has a plurality of two-dimensionally arrayed pixels, each ofthe pixels having a first photoelectric conversion portion that receivesa light beam passing through a first pupil region of an exit pupil of animaging optical system and a second photoelectric conversion portionthat receives a light beam passing through a second pupil region of theexit pupil of the imaging optical system different from the first pupilregion, the method comprising: generating a first image signal byconnecting, in a pupil divided direction, a first combined signalobtained by combining a signal of the first photoelectric conversionportion to a signal of another neighboring first photoelectricconversion portion, and generating a second image signal by connecting,in the pupil divided direction, a second combined signal obtained bycombining a signal of the second photoelectric conversion portion to asignal of another neighboring second photoelectric conversion portion;and detecting a phase difference between the first image signal and thesecond image signal, wherein, in the generating, in a case of combininga signal of the first photoelectric conversion portion to a signal ofanother neighboring first photoelectric conversion portion, or combininga signal of the second photoelectric conversion portion to a signal ofanother neighboring second photoelectric conversion portion, weightingof a signal of a photoelectric conversion portion in which an effect ofcrosstalk from a neighboring photoelectric conversion portion is largeis decreased, and combining is performed.

According to a third aspect of the present invention, there is providedan image capturing apparatus comprising: an image sensor that has aplurality of two-dimensionally arrayed pixels, each of the pixels havinga first photoelectric conversion portion that receives a light beampassing through a first pupil region of an exit pupil of an imagingoptical system and a second photoelectric conversion portion thatreceives a light beam passing through a second pupil region of the exitpupil of the imaging optical system different from the first pupilregion; and at least one processor or circuit configured to perform theoperations of the following units: a generation unit configured togenerate a first image signal, in a pupil divided direction, based on afirst signal obtained by combining a signal of the first photoelectricconversion portion to a signal of another neighboring firstphotoelectric conversion portion, and generate a second image signal, inthe pupil divided direction, based on a second signal obtained bycombining a signal of the second photoelectric conversion portion to asignal of another neighboring second photoelectric conversion portion;and a focus detection unit configured to detect a phase differencebetween the first image signal and the second image signal, wherein, ina case of combining a signal of the first photoelectric conversionportion to a signal of another neighboring first photoelectricconversion portion, or combining a signal of the second photoelectricconversion portion to a signal of another neighboring secondphotoelectric conversion portion, the generation unit decreasesweighting of a signal of a photoelectric conversion portion in which aneffect of crosstalk from a neighboring photoelectric conversion portionis large, and performs combining.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a functionalconfiguration of an image capturing apparatus according to an embodimentof the present invention.

FIGS. 2A and 2B are diagrams showing a configuration example of an imagesensor in an embodiment.

FIG. 3 is a diagram showing a configuration example of an image sensorin an embodiment.

FIG. 4 is a timing chart showing an exemplary operation of the imagesensor in FIG. 3.

FIG. 5A is a diagram showing the relationship between photoelectricconversion regions and an exit pupil in an embodiment.

FIG. 5B is a diagram showing the relationship between photoelectricconversion regions and an exit pupil in an embodiment.

FIG. 6 is a diagram showing the property of light receiving sensitivitydistribution of a photoelectric conversion portion with respect to anincident angle in an embodiment.

FIGS. 7A to 7C are diagrams for describing pupil regions correspondingto photoelectric conversion portions at a peripheral image height of animage sensor.

FIG. 8 is a diagram showing an example of a shooting range and focusdetection regions in an embodiment.

FIG. 9 is a diagram showing pixels arranged within a focus detectionregion in an embodiment.

FIG. 10A is a diagram showing line spread function of pixel outputs foreach color filter in an embodiment.

FIG. 10B is a diagram showing line spread function of pixel output foreach color filter in an embodiment.

FIGS. 11A and 11B are diagrams for describing crosstalk between pixelsin an embodiment.

FIG. 12 is a flowchart showing a focus adjustment operation in anembodiment.

FIG. 13 is a flowchart showing an example of a reliability determinationmethod in an embodiment.

FIG. 14 is a diagram showing a difference between two signals in anembodiment.

FIG. 15 is a flowchart showing an example of a method for calculating adefocus amount in an embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described indetail below with reference to the attached drawings. Here, embodimentswill be described in which a focus detection apparatus according to thepresent invention is applied to an interchangeable-lens digitalsingle-lens reflex camera (a camera system). However, the presentinvention can be applied to any electronic devices having an imagesensor that can generate signals to be used in focus detection of aphase difference detection method. Such electronic devices includegeneral cameras such as digital still cameras and digital video cameras,and mobile phone devices, computer devices, media players, robotdevices, gaming devices, home electric appliances and the like that havea camera function, but there is no limitation thereto.

FIG. 1 is a diagram showing a configuration example of a camera systemconstituted by a camera with an interchangeable taking lens and a takinglens, as an embodiment of an image capturing apparatus of the presentinvention. In FIG. 1, the camera system is constituted by a camera 100and an interchangeable taking lens 300.

A light beam passing through the taking lens 300 passes through a lensmount 106, is reflected upward by a main mirror 130, and is incident toan optical finder 104. The optical finder 104 makes it possible for aphotographer to shoot a subject while observing a subject optical image.Some functions of a display unit 54, for example, in-focus indication,camera shake alert display, aperture value display, exposure correctiondisplay, and the like, which will be described later, are installed inthe optical finder 104.

A portion of the main mirror 130 is made by a semi-transmissive halfmirror, and a portion of a light beam that is incident to the mainmirror 130 passes through this half mirror portion, is reflecteddownward by a sub mirror 131, and is incident to a focus detectionapparatus 105. The focus detection apparatus 105 is a focus detectionapparatus that adopts a phase difference detection method, and that hasa secondary imaging optical system and a line sensor, and outputs a pairof image signals to an AF unit (autofocus unit) 42. In the AF unit 42,phase difference detection calculation is performed on a pair of imagesignals, and the defocus amount and the defocus direction of the takinglens 300 are obtained. Based on this calculation result, a systemcontrol unit 50 causes a focus control unit 342 (to be described later)of the taking lens 300 to perform driving control of a focus lens.

In the case of performing still image shooting, electronic finderdisplay, or moving image shooting when focus adjustment processing ofthe taking lens 300 ends, the main mirror 130 and the sub mirror 131 areretracted from the light path using a quick return mechanism (notillustrated). In this case, a light beam passing through the taking lens300, and is incident to the camera 100 can enter an image sensor 14 viaa shutter 12 for controlling the exposure light amount. After a shootingoperation performed by the image sensor 14 ends, the main mirror 130 andthe sub mirror 131 return to positions as illustrated.

The image sensor 14 is a CCD or CMOS image sensor, and has aconfiguration in which a plurality of pixels that have photoelectricconversion regions (or photodiodes) are two-dimensionally arranged. Theimage sensor 14 outputs electrical signals corresponding to a subjectoptical image. Electrical signals obtained by the image sensor 14performing photoelectric conversion are sent to an A/D converter 16, andanalog signal outputs are converted into digital signals (image data).Note that the A/D converter 16 may be incorporated in the image sensor14 as will be described later.

The image sensor 14 in this embodiment is configured such that at leastsome pixels have a plurality of photoelectric conversion regions (orphotodiodes). As described above, pixels having such a configuration canoutput signals that are used for focus detection of a phase differencedetection method. Therefore, even in a case where the main mirror 130and the sub mirror 131 retreat from the light path due to the quickreturn mechanism, and light does not enter the focus detection apparatus105, it is possible to perform focus detection of a phase differencedetection method using outputs of the image sensor 14.

A timing generation circuit 18 supplies clock signals and controlsignals to the image sensor 14, the A/D converter 16, and a D/Aconverter 26. The timing generation circuit 18 is controlled by a memorycontrol unit 22 and the system control unit 50. The system control unit50 controls the timing generation circuit 18 so as to supply, to theimage sensor 14, control signals for reading out outputs of somephotoelectric conversion regions from the pixels that have a pluralityof photoelectric conversion regions, and additively reading outputs ofall the photoelectric conversion regions.

An image processing unit 20 applies predetermined processing such aspixel interpolation processing, white balance adjustment processing, andcolor conversion processing to image data from the A/D converter 16 orimage data from the memory control unit 22.

The image processing unit 20 also generates a pair of sequences ofsignals that are used for focus detection of a phase differencedetection method, from output signals that are used for generatingsignals for focus detection, out of image data from the A/D converter 16(output signals of the image sensor 14). After that, the pair ofsequences of signals are sent to the AF unit 42 via the system controlunit 50. The AF unit 42 detects a deviation amount (shift amount)between the sequences of signals by calculating a correlation betweenthe pair of sequences of signals, and converts the deviation amount intoa defocus amount and defocus direction of the taking lens 300. The AFunit 42 outputs the defocus amount and defocus direction after theconversion to the system control unit 50. The system control unit 50drives the focus lens through the focus control unit 342 of the takinglens 300, and adjusts the focal distance of the taking lens 300.

In addition, the image processing unit 20 can calculate a contrastevaluation value based on signals for generating normal image data(corresponding to the above-described A+B signal) that is obtained fromthe image sensor 14. The system control unit 50 performs shooting usingthe image sensor 14 while changing the focus lens position through thefocus control unit 342 of the taking lens 300, and examines a change inthe contrast evaluation value calculated by the image processing unit20. The system control unit 50 then drives the focus lens to a positionat which the contrast evaluation value is the largest. The camera 100 ofthis embodiment can also perform focus detection by a contrast detectionmethod in this manner.

Therefore, even when the main mirror 130 and the sub mirror 131 haveretreated to the outside of the light path, such as during live viewdisplay and moving image shooting, the camera 100 can perform focusdetection using both a phase difference detection method and a contrastdetection method based on signals obtained from the image sensor 14.Also, in normal still image shooting in which the main mirror 130 andthe sub mirror 131 are in the light path, in the camera 100, the focusdetection apparatus 105 can perform focus detection of a phasedifference detection method. In this manner, the camera 100 can performfocus detection in any state, for example, during still image shooting,live view display, and moving image shooting.

The memory control unit 22 controls the A/D converter 16, the timinggeneration circuit 18, the image processing unit 20, an image displaymemory 24, the D/A converter 26, a memory 30, and acompression/decompression unit 32. Data in the A/D converter 16 is thenwritten to the image display memory 24 or the memory 30 via the imageprocessing unit 20 and the memory control unit 22, or only via thememory control unit 22. Image data that is to be displayed and iswritten in the image display memory 24 is displayed on an image displayunit 28 constituted by a liquid crystal monitor or the like, via the D/Aconverter 26. By sequentially displaying a moving image shot using theimage sensor 14 on the image display unit 28, an electronic finderfunction (live view display) can be realized. The image display unit 28can turn on/off display according to an instruction of the systemcontrol unit 50, and in the case where display is turned off, powerconsumption of the camera 100 can be reduced significantly.

Moreover, the memory 30 is used for temporarily storing still images andmoving images that have been shot, and has a sufficient storage capacityfor storing a predetermined number of still images and a moving image ofa predetermined time. This makes it possible to write a large amount ofimage data to the memory 30 at a high speed even in a case of continuousshooting or panoramic shooting. The memory 30 can also be used as a workarea of the system control unit 50. The compression/decompression unit32 has a function for compressing and decompressing image data throughAdaptive Discrete Cosine Transform (ADCT) or the like, and reads imagesstored in the memory 30, performs compression processing ordecompression processing, and writes the processed image data back tothe memory 30.

A shutter control unit 36 controls the shutter 12 based on photometryinformation from a photometry unit 46, in cooperation with a diaphragmcontrol unit 344 that controls a diaphragm 312 of the taking lens 300.An interface unit 38 and a connector 122 electrically connect the camera100 and the taking lens 300 to each other. The interface unit 38 and theconnector 122 have a function for transmitting control signals, statesignals, data signals, and the like between the camera 100 and thetaking lens 300, and also supplying currents of various voltages. Inaddition, a configuration may be adopted in which such signals aretransmitted through not only electric communication but also opticalcommunication, sound communication and the like.

The photometry unit 46 performs automatic exposure control (AE)processing. The luminance of a subject optical image can be measured byallowing a light beam passing through the taking lens 300 to enter thephotometry unit 46 via the lens mount 106, the main mirror 130, and aphotometry lens (not illustrated). The photometry unit 46 can determineexposure conditions using a program diagram in which subject luminancesand exposure conditions are associated with each other, and the like.Also, the photometry unit 46 has a dimming processing function incooperation with a flash 48. Note that the system control unit 50 canalso cause the shutter control unit 36 and the diaphragm control unit344 of the taking lens 300 to perform AE control, based on a result ofthe image processing unit 20 calculating image data of the image sensor14. The flash 48 has a light projecting function for an AF auxiliarylight and a flash adjusting function.

The system control unit 50 has a programmable processor such as a CPU oran MPU, and controls overall operations of a camera system by executinga program stored in advance. A nonvolatile memory 52 stores constants,variables, programs and the like for operating the system control unit50. For example, the display unit 54 is a liquid crystal displayapparatus that displays an operation state, a message, and the likeusing characters, an image, sound, and the like according to the systemcontrol unit 50 executing a program. One or more display units 54 areinstalled at positions near the operation unit of the camera 100 atwhich it is easy to visually recognize the display units 54, and areeach constituted by a combination of an LCD, LED, and the like. Displaycontents that are displayed on the LCD or the like from among displaycontents of the display unit 54 include information regarding the numberof shooting images such as the number of images to be recorded and theremaining number of images that can be shot, and information regardingshooting conditions such as shutter speed, aperture value, exposurecorrection, and flash. In addition, battery remaining capacity, time anddate, and the like are also displayed. Moreover, some functions of thedisplay unit 54 are installed in the optical finder 104, as describedabove.

A nonvolatile memory 56 is an electrically erasable/recordable memory,and an EEPROM is used as the nonvolatile memory 56, for example.Reference numerals 60, 62, 64, 66, 68 and 70 indicate operation unitsfor inputting various operation instructions of the system control unit50, and are constituted by one or more combinations of switches, dials,a touch panel, pointing through sight line detection, a soundrecognition apparatus, and the like.

A mode dial 60 can switch and set function modes such as power sourceoff, an automatic shooting mode, a manual shooting mode, a playbackmode, and a PC connection mode. A shutter switch SW1 indicated byreference numeral 62 is turned on when a shutter button (notillustrated) is half-pressed, and instructs operation start of AFprocessing, AE processing, AWB processing, EF processing, and the like.A shutter switch SW2 indicated by reference numeral 64 is turned on whenthe shutter button is fully pressed, and instructs operation start of aseries of processing related to shooting. A series of processing relatedto shooting refers to exposure processing, developing processing,recording processing, and the like. In exposure processing, signals thathave been read out from the image sensor 14 are written as image data tothe memory 30 via the A/D converter 16 and the memory control unit 22.In developing processing, development using calculation performed by theimage processing unit 20 and the memory control unit 22 is performed. Inrecording processing, image data is read out from the memory 30, iscompressed by the compression/decompression unit 32, and is written asimage data to a recording medium 150 or 160.

An image display ON/OFF switch 66 can set ON/OFF of the image displayunit 28. This function makes it possible to save electricity by cuttingoff current supply to the image display unit 28 constituted by a liquidcrystal monitor or the like when performing shooting using the opticalfinder 104. A quick review ON/OFF switch 68 sets a quick review functionfor automatically reproducing shot image data immediately aftershooting. An operation unit 70 is constituted by various buttons, atouch panel, and the like. The various buttons include a menu button, aflash setting button, a single shooting/continuous shooting/self-timerswitching button, an exposure correction button, and the like.

A power source control unit 80 is constituted by a battery detectioncircuit, a DC/DC converter, a switch circuit for switching a block thatis energized, and the like. Whether or not a battery is mounted, thetype of battery, and battery remaining capacity are detected, and theDC/DC converter is controlled based on a detection result and aninstruction of the system control unit 50, and a necessary voltage issupplied, for a necessary period, to constituent elements that include arecording medium. Connectors 82 and 84 connect, to the camera 100, apower source unit 86 constituted by a primary battery such as analkaline battery or a lithium battery, a secondary battery such as aNiCd battery, a NiMH battery, or a lithium ion battery, an AC adapter,and the like.

Interfaces 90 and 94 have a function for connecting to a recordingmedium such as a memory card or a hard disk, and connectors 92 and 96physically connect to a recording medium such as a memory card or a harddisk. A recording medium mounting/dismounting detection unit 98 detectswhether or not a recording medium is mounted on the connector 92 or 96.Note that, in this embodiment, description is given in which twointerfaces and two connectors for mounting a recording medium areprovided, but a configuration may be adopted in which one or moreinterfaces and one or more connectors, or any number of interfaces andany number of connectors are provided. Also, a configuration may beadopted in which interfaces and connectors of different standards areprovided in combination. Furthermore, it is possible to transfer imagedata and administrative information attached to image data to/fromanother peripheral device such as a computer or a printer by connectingvarious communication cards such as a LAN card to the interface andconnector.

A communication unit 110 has various communication functions such aswired communication and wireless communication. A connector 112 connectsthe camera 100 to another device using the communication unit 110, andis an antenna in the case of wireless communication. The recording media150 and 160 are memory cards, hard disks, or the like. The recordingmedia 150 and 160 include recording units 152 and 162 constituted by asemiconductor memory, a magnetic disk, or the like, interfaces 154 and164 to the camera 100, and connectors 156 and 166 for connecting to thecamera 100.

Next, the taking lens 300 will be described. The taking lens 300 ismechanically and electrically connected to the camera 100 by engaging alens mount 306 with the lens mount 106 of the camera 100. Electricalconnection is realized by the connector 122 and a connector 322respectively provided on the lens mount 106 and the lens mount 306. Alens 311 includes a focus lens for adjusting the focal distance of thetaking lens 300. The focus control unit 342 performs focus adjustment ofthe taking lens 300 by driving the focus lens along the optical axis.The system control unit 50 controls operations of the focus control unit342 through a lens system control unit 346. The diaphragm 312 adjuststhe amount and angle of subject light that is incident to the camera100.

The connector 322 and an interface 338 electrically connect the takinglens 300 to the connector 122 of the camera 100. The connector 322 has afunction for transmitting control signals, state signals, data signals,and the like between the camera 100 and the taking lens 300, and alsohas a function for supplying currents of various voltages. The connector322 may be configured to transmit such signals through not only electriccommunication but also optical communication, sound communication, andthe like.

A zoom control unit 340 drives a variable magnification lens of the lens311 so as to adjust the focal distance (field angle) of the taking lens300. If the taking lens 300 is a single-focal lens, the zoom controlunit 340 does not exist. The diaphragm control unit 344 controls thediaphragm 312 in cooperation with the shutter control unit 36 thatcontrols the shutter 12 based on photometry information from thephotometry unit 46.

The lens system control unit 346 has a programmable processor such as aCPU or an MPU, and controls overall operations of the taking lens 300 byexecuting a program stored in advance. Also, the lens system controlunit 346 has a function of a memory that stores constants, variables,programs, and the like for operating the taking lens. A nonvolatilememory 348 stores identification information such as a number unique toa taking lens, administrative information, function information such asopen aperture value, minimum aperture value, focal distance, present andpast setting values, and the like.

In this embodiment, lens frame information that is based on the state ofthe taking lens 300 is also stored. This lens frame information includesinformation regarding the radius of a frame opening for defining a lightbeam that passes through the taking lens and information regarding thedistance from the image sensor 14 to the frame opening. The diaphragm312 is included in the frame that defines a light beam that passesthrough the taking lens, and, in addition, an opening of a lens framepart that holds a lens, or the like corresponds to the frame. Inaddition, the frame for defining a light beam that passes through thetaking lens changes according to the focus position and zoom position ofthe lens 311, and thus a plurality of pieces of lens frame informationare prepared in correspondence with focus positions and zoom positionsof the lens 311. When the camera 100 performs focus detection using afocus detection means, optimum lens frame information corresponding tothe focus position and zoom position of the lens 311 is selected, and issent to the camera 100 through the connector 322.

The above is the configuration of the camera system of this embodimentconstituted by the camera 100 and the taking lens 300.

Next, the configuration of the image sensor 14 will be described withreference to FIGS. 2A, 2B and 3.

FIG. 2A is a diagram showing an example of a circuit configuration of apixel that can output a signal to be used for focus detection of a phasedifference detection method from among a plurality of pixels of theimage sensor 14. Here, a configuration will be described in which twophotodiodes PD201 a and 201 b are provided in one pixel 200 as aplurality of photoelectric conversion regions or photoelectricconversion portions that share one microlens. However, more (e.g., four)photodiodes may be provided for one microlens. In addition, thearrangement of photodiodes is not limited to the arrangement only in thehorizontal direction, and the photodiodes may be arranged in thevertical direction. As will be described later, the photodiode 201 a andthe photodiode 201 b function as focus detection pixels, and alsofunction as imaging pixels.

Transfer switches 202 a and 202 b, a reset switch 205, and a selectionswitch 206 are each constituted by a MOS transistor, for example. In thefollowing description, these switches are N-type MOS transistors, butmay be P-type MOS transistors, or may be other switching elements.

FIG. 2B is a diagram schematically showing n pixels in a horizontaldirection and m pixels in a vertical direction from among a plurality ofpixels arrayed two-dimensionally in the image sensor 14. Here, all ofthe pixels have the configuration shown in FIG. 2A. A microlens 236 isprovided in each pixel, and the two photodiodes 201 a and 201 b arearranged for one microlens. Hereinafter, a signal that is obtained fromthe photodiode 201 a is referred to as an A signal or a first signal,and a signal that is obtained from the photodiode 201 b is referred toas a B signal or a second signal. In addition, a sequence of signals forfocus detection that are generated from a plurality of A signals arereferred to as an A image or first image signals, and a sequence ofsignals for focus detection that are generated from a plurality of Bsignals are referred to as a B image or second image signals. Inaddition, a pair of an A image and a B image are referred to as a pairof sequences of signals or a pair of image signals.

The transfer switch 202 a is connected between the photodiode 201 a anda floating diffusion portion (hereinafter, FD) 203. In addition, thetransfer switch 202 b is connected between the photodiode 201 b and FD203. The transfer switches 202 a and 202 b are elements for respectivelytransferring electric charges generated in the photodiodes 201 a and 201b to common FD 203. The transfer switches 202 a and 202 b arerespectively controlled using control signals TX_A and TX_B.

The floating diffusion portion (FD) 203 temporarily holds electriccharges transferred from the photodiodes 201 a and 201 b, and functionsas a charge/voltage conversion unit (capacitor) that converts heldelectric charges into a voltage signal.

An amplification unit 204 is a source follower MOS transistor. The gateof the amplification unit 204 is connected to FD 203, and the drain ofthe amplification unit 204 is connected to a common power source 208that supplies a power source potential VDD. The amplification unit 204amplifies the voltage signal that is based on electric charges held inFD 203, and outputs the signal as an image signal.

The reset switch 205 is connected between FD 203 and the common powersource 208. The reset switch 205 is controlled by a control signal RES,and has a function for resetting the potential of FD 203 to the powersource potential VDD of the common power source 208.

The selection switch 206 is connected between the source of theamplification unit 204 and a vertical output line 207. The selectionswitch 206 is controlled by a control signal SEL, and outputs an imagesignal amplified by the amplification unit 204 to the vertical outputline 207.

FIG. 3 is a diagram showing a configuration example of the image sensor14. The image sensor 14 has a pixel array 234, a vertical scanningcircuit 209, current source loads 210, readout circuits 235, commonoutput lines 228 and 229, a horizontal scanning circuit 232, and a dataoutput unit 233. In the following description, all of the pixelsincluded in the pixel array 234 have the circuit configuration shown inFIG. 2A. However, some pixels may have a configuration in which onephotodiode is provided for one microlens.

The pixel array 234 has a plurality of pixels 200 arranged in a matrix.FIG. 3 shows the pixel array 234 having four rows and n columns for easeof description. However, the number of rows and the number of columns ofthe pixels 200 of the pixel array 234 may be appropriately set. Inaddition, in this embodiment, the image sensor 14 is a single-platecolor image sensor, and has Bayer array primary color filters.Therefore, one of red (R), green (G) and blue (B) color filters isprovided for each of the pixels 200. Note that color and arrangementconfigurations of the color filters are not limited particularly. Inaddition, some pixels included in the pixel array 234 are shielded fromlight so as to form an optical black (OB) region.

The vertical scanning circuit 209 supplies various control signals shownin FIG. 2A to the pixels 200 in each of the rows via the drive signalline 208 provided for the row. Note that, in FIG. 3, for ease ofdescription, the drive signal line 208 in each row is indicated by aline, but, in actuality, there are a plurality of drive signal lines ineach row.

The pixels included in the pixel array 234 are connected, for eachcolumn, to the vertical output line 207 shared by the pixels in thecolumn. The current source load 210 is connected to each vertical outputline 207. Signals from the pixels 200 are input to the readout circuits235 provided for the respective columns, through the vertical outputlines 207.

The horizontal scanning circuit 232 outputs control signals hsr(0) tohsr(n−1) respectively corresponding to the readout circuits 235. One ofn readout circuits 235 is selected using a control signal hsr ( ). Thereadout circuit 235 selected using the control signal hsr ( ) outputs asignal to the data output unit 233 through the common output lines 228and 229.

Next, an example of a specific circuit configuration of the readoutcircuit 235 will be described. FIG. 3 shows an example of the circuitconfiguration of one of the n readout circuits 235, but the otherreadout circuits 235 have the same configuration. The readout circuit235 of this embodiment includes a ramp A/D converter.

A signal that has been input to the readout circuit 235 through thevertical output line 207 is input to an inverting input terminal of anoperational amplifier 213 via a clamp capacitor 211. A reference voltageVref is supplied from a reference voltage source 212 to a non-invertinginput terminal of the operational amplifier 213. Feedback capacitors 214to 216 and switches 218 to 220 are connected between the inverting inputterminal and output terminal of the operational amplifier 213. A switch217 is further connected between the inverting input terminal and outputterminal of the operational amplifier 213. The switch 217 is controlledby a control signal RES_C, and has a function for causing the two endsof each of the feedback capacitors 214 to 216 to short-circuit. Inaddition, the switches 218 to 220 are controlled using control signalsGAIN0 to GAIN2 by the system control unit 50.

An output signal of the operational amplifier 213 and a ramp signal 224that is output from a ramp signal generator 230 are input to acomparator 221. Latch_N222 is a storage element for holding a noiselevel (N signal), and Latch_S is a storage element for holding a signallevel (A signal) and a signal level (A+B signal) acquired by adding an Asignal and a B signal. Output of the comparator 221 (a value indicatinga comparison result) and output 225 of a counter 231 (a counter value)are respectively input to Latch_N222 and Latch_S223. Operations ofLatch_N222 and Latch_S223 (whether Latch_N222 and Latch_S223 are enabledor disabled) are respectively controlled by control signals LATEN_N andLATEN_S. A noise level held in Latch_N222 is output to the common outputline 228 via a switch 226. A signal level held in Latch_S223 is outputto the common output line 229 via a switch 227. The common output lines228 and 229 are connected to the data output unit 233.

The switches 226 and 227 are controlled by a control signal hsr(h) fromthe horizontal scanning circuit 232. Here, h indicates the column numberof the readout circuit 235 to which the control signal line isconnected. Signal levels held in Latch_N222 and Latch_S223 of each ofthe readout circuits 235 are sequentially output to common output lines238 and 229, and are output to the memory control unit 22 and the imageprocessing unit 20 through the data output unit 233. This operation forsequentially outputting signal levels held in the readout circuits 235to the outside is called horizontal transferring. Note that controlsignals (except for hsr ( )) that are input to the readout circuits andcontrol signals of the vertical scanning circuit 209, the horizontalscanning circuit 232, the ramp signal generator 230, and the counter 231are supplied from the timing generation circuit 18 and the systemcontrol unit 50.

A readout operation for pixels of one row will be described withreference to FIG. 4 that is a timing chart related to a readoutoperation of the image sensor 14 shown in FIG. 3. Note that, when acontrol signal is at H, the switch is on, and when a control signal isat L, the switch is off.

At a time t1, the vertical scanning circuit 209 changes the controlsignals TX_A and TX_B from L to H in the state where the control signalRES is set to H, and turns on the transfer switches 202 a and 202 b.Accordingly, electric charges accumulated in the photodiodes 201 a and201 b are transferred to the power source 208 via the transfer switches202 a and 202 b and the reset switch 205, and the photodiodes 201 a and201 b are reset. Also, FD 203 is reset similarly. At a time t2, when thevertical scanning circuit 209 changes the control signals TX_A and TX_Bto L, and turns off the transfer switches 202 a and 202 b, accumulationof photocharges in the photodiodes 201 a and 201 b is started.

When a predetermined accumulation time has elapsed, the verticalscanning circuit 209 changes the control signal SEL to H at a time t3,and turns on the selection switch 206. Accordingly, the source of theamplification unit 204 is connected to the vertical output line 207. Ata time t4, the vertical scanning circuit 209 changes the control signalRES to L, and turns off the reset switch 205. Accordingly, reset of FD203 is cancelled, and the reset signal level of FD 203 is read out tothe vertical output line 207 via the amplification unit 204, and isinput to the readout circuit 235.

After that, at a time t5, the timing generation circuit 18 changes thecontrol signal RES_C to L. Accordingly, the switch 217 is turned on, anda voltage that is based on the difference between the reset signal levelthat has been read out to the vertical output line 207 and the referencevoltage Vref is output from the operational amplifier 213. The imagesensor 14 is set in advance, based on an ISO sensitivity that has beenset using the operation unit 70, such that the system control unit 50changes one of the control signals GAIN0 to GAIN2 to H. For example, ifone of ISO sensitivities 100, 200, and 400 can be set in the camera 100of an embodiment, in the case of the ISO sensitivity 100, the controlsignal GAIN0 is at H, and the control signals GAIN1 and GAIN2 are at L.Similarly, in the case of the ISO sensitivity 200, the control signalGAIN1 is at H, and in the case of the ISO sensitivity 400, the controlsignal GAIN2 is at H. Note that a type of setting sensitivity and arelationship between setting sensitivities and control signals are notlimited thereto.

The operational amplifier 213 amplifies a voltage that has been input,with an inverted gain that is determined according to the capacity ratioof the clamp capacitor 211 to one of the feedback capacitors 214 to 216corresponding to a switch corresponding to a control signal that is at Hfrom among the control signals GAIN0 to GAIN2, and outputs the amplifiedvoltage. In this amplification, random noise components that occur incircuits before the operational amplifier 213 are also amplified.Therefore, the magnitude of random noise included in a signal afteramplification depends on the ISO sensitivity.

Next, at a time t6, the ramp signal generator 230 starts outputting aramp signal whose signal level increases linearly over time, and at thesame time, the counter 231 starts counting-up from a reset state. Inaddition, the timing generation circuit 18 changes LATEN_N to H, andenables Latch_N. The comparator 221 compares an output signal of theoperational amplifier 213 and the ramp signal that is output from theramp signal generator 230. When a ramp signal level exceeds the outputsignal level of the operational amplifier 213, output of the comparator221 changes from L to H (time t7). When the output of the comparator 221changes from L to H in the state where LATEN_N is at H, Latch_N222stores a counter value that is being output by the counter 231 at thispoint. The counter value stored in Latch_N222 is equivalent to a digitalvalue (N signal data) indicating an N signal level. Note that LATEN_S isat L, and thus Latch_S223 is disabled, and does not store the countvalue. After that, at a time t8, when the ramp signal level reaches apredetermined value, the ramp signal generator 230 stops outputting theramp signal, and the timing generation circuit changes LATEN_N to L.

At a time t9, the vertical scanning circuit 209 changes the controlsignal TX_A to H. Accordingly, the transfer switch 202 a is turned on,and photocharges (A signals) accumulated in the photodiode 201 a sincethe time t2 are transferred to FD 203. After that, at a time t10, thevertical scanning circuit 209 changes the control signal TX_A to L. FD203 converts the transferred electric charges into a potential, and thispotential (A signal level) is output to the readout circuit 235 via theamplification unit 204 and the vertical output line 207. The operationalamplifier 213 outputs a voltage that is based on the difference betweenthe A signal level that has been read out to the vertical output line207 and the reference voltage Vref. An inverted gain of the operationalamplifier 213 is determined according to the rate of the clamp capacitor211 to one of the feedback capacitors 214 to 216.

Next, at a time t11, the ramp signal generator 230 starts outputting aramp signal, and at the same time, the counter 231 starts counting-upfrom a reset state. In addition, the timing generation circuit 18changes LATEN_S to H, and enables Latch_S. The comparator 221 comparesan output signal of the operational amplifier 213 with the ramp signalthat is output by the ramp signal generator 230. When the ramp signallevel exceeds the output signal level of the operational amplifier 213,output of the comparator 221 changes from L to H (at a time t12). Whenoutput of the comparator 221 changes from L to H in the state wereLATEN_S is at H, Latch_S223 stores the counter value that is beingoutput by the counter 231 at this point. The counter value stored inLatch_S223 is equivalent to a digital value (A signal data) indicatingthe A signal level. Note that LATEN_N is at L, and thus Latch_N222 isdisabled, and does not store the count value. After that, at a time t13,when the ramp signal level reaches a predetermined value, the rampsignal generator 230 stops outputting the ramp signal, and the timinggeneration circuit changes LATEN_S to L.

After that, during a period from a time t14 to a time t15, thehorizontal scanning circuit 232 sequentially changes the control signalhsr(h) to H for a certain period. Accordingly, the switches 226 and 227of each of the readout circuits 235 are turned on for a certain period,and are then returned to off. N signal data and A signal data held inLatch_N222 and Latch_S223 of each of the readout circuits 235 arerespectively read out to the common output lines 228 and 229, and areinput to the data output unit 233. Regarding the A signal data and Nsignal data that have been output from each of the readout circuits 235,the data output unit 233 outputs a value acquired by subtracting the Nsignal data from the A signal data to the outside.

During a period from a time t16 to a time t17, the vertical scanningcircuit 209 changes the control signals TX_A and TX_B to H, and turns onthe transfer switches 202 a and 202 b. Accordingly, photocharges aretransferred from both the photodiodes 201 a and 201 b to FD 203. FD 203converts the transferred electric charges into a potential, and thispotential (A+B signal level) is output to the readout circuit 235 viathe amplification unit 204 and the vertical output line 207. Theoperational amplifier 213 outputs a voltage that is based on thedifference between the A+B signal level that has been read out to thevertical output line 207 and the reference voltage Vref.

Next, at a time t18, the ramp signal generator 230 starts outputting aramp signal, and, at the same time, the counter 231 starts counting-upfrom a reset state. In addition, the timing generation circuit 18changes LATEN_S to H, and enables Latch_S. The comparator 221 comparesan output signal of the operational amplifier 213 and the ramp signalthat is output by the ramp signal generator 230. When the ramp signallevel exceeds the output signal level of the operational amplifier 213,output of the comparator 221 changes from L to H (at a time t19). Whenoutput of the comparator 221 changes from L to H in the state whereLATEN_S is at H, Latch_S223 stores a counter value that is being outputby the counter 231 at this point. The counter value stored in Latch_S223is equivalent to a digital value (A+B signal data) indicating an A+Bsignal level. After that, at a time t20, when the ramp signal levelreaches a predetermined value, the ramp signal generator 230 stopsoutputting the ramp signal, and the timing generation circuit changesLATEN_S back to L.

After that, during a period from a time t21 to a time t22, thehorizontal scanning circuit 232 sequentially changes the control signalhsr(h) to H for a certain period. Accordingly, the switches 226 and 227of each of the readout circuits 235 are turned on for a certain period,and are then returned to off. The N signal data and A+B signal data heldin Latch_N222 and Latch_S223 of each of the readout circuits 235 arerespectively read out to the common output lines 228 and 229, and areinput to the data output unit 233. Regarding the A+B signal data and Nsignal data that have been output from each of the readout circuits 235,the data output unit 233 outputs a value acquired by subtracting the Nsignal data from the A+B signal data to the outside.

When the timing generation circuit 18 changes the control signal RES_Cto H at the time t22, the vertical scanning circuit 209 changes thecontrol signal RES to H at a time t23, and the vertical scanning circuit209 changes the control signal SEL to L at a time t24, a readoutoperation for one row is complete. By repeating this operation for apredetermined number of rows, image signals for one screen can beacquired.

In this manner, it is possible to read out A signals and A+B signalsfrom which reset noise has been removed, from the image sensor 14. Asignals are used as signals for focus detection, and A+B signals areused as signals for forming a captured image. A+B signals and A signalsare also used for generating B signals for focus detection.

Note that the image sensor 14 of this embodiment has two types ofreadout modes, namely an all-pixel readout mode and a thinned readoutmode. The all-pixel readout mode is a mode for reading out all of theeffective pixels, and, for example, is set when obtaining ahigh-definition still image.

The thinned readout mode is a mode for reading out a smaller number ofpixels than the all-pixel readout mode, and is set in the case ofobtaining an image whose resolution is lower than that of ahigh-definition still image, such as a moving image and an image forpreview, and in the case where it is necessary to perform readout at ahigh speed. For example, it is possible to thin pixels at the same ratioor different ratios in the horizontal and vertical directions of animage, and read out the thinned pixels. Note that “thinning” includesnot only a configuration for not performing readout itself, but also aconfiguration for discarding (ignoring) signals that have been read out,and a configuration for adding a plurality of signals that have beenread out and generating one signal. For example, by averaging signalsthat have been read out from a plurality of adjacent pixels andgenerating one signal, S/N can be improved.

FIG. 5A is a diagram for describing the conjugate relationship betweenthe exit pupil plane of the taking lens 300 and the photodiodes 201 aand 201 b of a pixel 200 (central pixel) arranged in the vicinity of thecenter of the imaging plane of the image sensor 14, in the imagecapturing apparatus of this embodiment. The photodiodes 201 a and 201 bin the image sensor 14 and the exit pupil plane of the taking lens 300are designed so as to have a conjugate relationship using an on-chipmicrolens 201 i. In addition, generally, the exit pupil plane of thetaking lens 300 substantially matches a plane on which an iris diaphragmfor adjusting the light amount is provided.

On the other hand, the taking lens 300 of this embodiment is a zoom lensthat has a magnification changing function. In some zoom lenses, thesize of the exit pupil and the distance (exit pupil distance) from theimaging plane to the exit pupil change when a magnification changingoperation is performed. FIG. 5A shows the state where the focal distanceof the taking lens 300 is at the center between the wide angle end andthe telephoto-end. Using an exit pupil distance D1 in this state as astandard value, the shape of the on-chip microlens and an eccentricparameter that is based on the image height (the distance from thecenter of the screen or XY coordinates) are optimally designed.

In FIG. 5A, the taking lens 300 has a first lens group 101, a barrelmember 101 b that holds the first lens group, a third lens group 105,and a barrel member 105 b that holds the third lens group. The takinglens 300 also has a diaphragm 102, an opening plate 102 a that definesthe opening diameter when the diaphragm is opened, and a diaphragm blade102 b for adjusting the opening diameter when stopping down the lens.Note that, in FIG. 5A, 101 b, 102 a, 102 b, and 105 b that act asmembers for restricting a light beam that passes through the taking lens300 denote optical virtual images when observed from the imaging plane.In addition, a synthetic aperture in the vicinity of the diaphragm 102is defined as an exit pupil of the taking lens 300, and the distancefrom the imaging plane to the exit pupil is defined as the exit pupildistance D1.

The photodiodes (photoelectric conversion portions) 201 a and 201 b arearranged in the lowermost layer of the pixel 200. Interconnect layers201 e to 201 g, a color filter 201 h, and the on-chip microlens 201 iare provided above the photodiodes 201 a and 201 b. The photodiodes 201a and 201 b are projected on the exit pupil plane of the taking lens 300by the on-chip microlens 201 i. In other words, the exit pupil isprojected on the surfaces of the photodiodes 201 a and 201 b via theon-chip microlens 201 i.

FIG. 5B shows projected images EP1 a and EP1 b of the photodiodes 201 aand 201 b on the exit pupil plane of the taking lens 300. A circle TLindicates the maximum incident range of a light beam that is incident tothe pixel 200, on the exit pupil plane, the maximum incident range beingdefined by the opening plate 102 a of the diaphragm 102. The circle TLis defined by the opening plate 102 a, and thus, in the drawings, thecircle TL is also indicated by reference numeral 102 a. FIG. 5B shows acentral pixel, and the vignetting of a light beam is symmetricalrelative to the optical axis, and the photodiodes 201 a and 201 breceive a light beam passing through pupil regions of the same size. Thecircle TL includes a large portion of the projected images EP1 a and EP1b, and thus there is substantially no vignetting of the light beam.Therefore, in the case where signals that have been photoelectricallyconverted by the photodiodes 201 a and 201 b are added, a result ofphotoelectrically converting a light beam passing through the circle TL,in other words, substantially the entirety of the exit pupil region isobtained. A region of the exit pupil from which a light beam is receivedby the photodiode 201 a is referred to as a first pupil region, a regionof the exit pupil from which a light beam is received by the photodiode201 b is referred to as a second pupil region, and a region acquired byadding the first pupil region and the second pupil region is referred toas a third pupil region.

Crosstalk that occurs between pixels will be described with reference tothe structure of the pixel 200 shown in FIGS. 5A and 5B. Crosstalkbetween adjacent pixels is classified into two categories according toits causes. One is optical crosstalk that occurs due to arrangement ofthe on-chip microlens 201 i, the color filter 201 h, and theinterconnect layers 201 e to 201 g, and stray light caused byreflection, scattering, and the like of transmission light, for example.The other is electric crosstalk that occurs due to movement of electriccharges generated in a photoelectric conversion portion of each pixel toanother pixel.

Regarding optical crosstalk, the amount of crosstalk (the amount oflight that traveled to adjacent pixels) that occurs due to difference intransmittance, reflectivity, refractive index, and the like according tothe wavelength of light also differs for each color. In addition, theamount of crosstalk that occurs also differs according to the incidentangle of light. Furthermore, due to the anisotropy of the interconnectlayers 201 e to 201 g of a pixel, anisotropy occurs also in a directionin which crosstalk occurs.

Similarly, also regarding electric crosstalk, depths at which crosstalkinvades photodiodes differ according to the wavelengths (depths at whichphotoelectrical conversion is performed), and thus the amount ofcrosstalk (the amount of electric charges that travel to adjacentpixels) that occurs also differs for each color. In addition, anisotropyoccurs in a direction in which electric crosstalk occurs, due to theanisotropy of the interconnect layers 201 e to 201 g of the pixel andthe height of a barrier against signal charges between pixels inaddition to the wavelength. The height of a barrier against signalcharges between pixels is described in Japanese Patent Laid-Open No.2014-187067 in detail.

The pixels of the image sensor of this embodiment are described assumingthat crosstalk has anisotropy, and the crosstalk amount is larger in thehorizontal direction than in the vertical direction. For example, such asituation is achieved in the case where the height of a barrier betweenpixels and the state of the interconnect layers are configured such thatcrosstalk is less likely to occur in the vertical direction than in thehorizontal direction.

In the case where the amount of crosstalk that occurs depends on thewavelength and incident angle, or includes anisotropy as describedabove, a focus detection error occurs. In this embodiment, a method thatcan realize accurate focus detection even if crosstalk occurs isproposed. Detailed description will be given later.

Next, the property of light receiving sensitivity distribution of thephotodiodes 201 a and 201 b with respect to the incident angle of lightfor each wavelength will be described with reference to FIG. 6. FIG. 6is a diagram showing the signal intensity distribution of thephotodiodes 201 a and 201 b of two pixels 200 with respect to theincident angle, and the two pixels 200 are arranged in the vicinity ofthe center of the imaging plane of the image sensor 14, and have red (R)and green (G) color filters. The horizontal axis indicates incidentangle, and positive incident angles are on the right. The vertical axisindicates signal intensity equivalent to the light receivingsensitivity, and the incident angle at an intersection C of the signalintensity between the photodiodes 201 a and 201 b is the origin. Theintensity distribution of the photodiode 201 a takes the maximum valueof intensity in the case where the incident angle is negative, and theintensity distribution of the photodiode 201 b takes the maximum valueof intensity in the case where the incident angle is positive. Due todifference in refractive index according to the wavelength, the shapesof properties of light receiving sensitivities of R and G with respectto the incident angle differ. Note that color filters arranged on thepixels of the image sensor 14 include blue (B) color filters, which areomitted here for ease of description. Note that, also in the case ofblue, the intensity distribution shape is different from those of redand green due to dependency on wavelength.

As being apparent from FIG. 6, at an incident angle C corresponding tothe intersection of the property curve, signal intensity ratios betweenR and G are equal in the photodiodes 201 a and 201 b. On the other hand,a change rate of signal intensity for the incident angle differs betweenR and G, and thus, in a region separated from the incident angle C, thesignal intensity ratios between R and G differ between the photodiodes201 a and 201 b. For example, in the case where the incident angle isnegative, in the photodiode 201 a, the signal intensity of R is smallerthan the signal intensity of G. On the other hand, in the photodiode 201b, the signal intensity of R is larger than the signal intensity of G.In this case, the signal intensity ratios between R and G differ betweena pair of signals that are used for focus detection.

A case where signal intensity ratio differs according to each wavelengthwill be described with reference to FIGS. 7A to 7C. FIGS. 7A to 7C arediagrams showing the relationship between a first pupil region 501corresponding to the photodiode 201 a, a second pupil region 502corresponding to the photodiode 201 b, and the third pupil region 400that is the exit pupil of the taking lens 300, at a peripheral imageheight of the image sensor 14. The horizontal direction of the thirdpupil region 400 is referred to as an X axial direction, and thevertical direction is referred to as a Y axial direction.

FIG. 7A is a diagram showing the case where the exit pupil distance D1of the taking lens 300 and a set pupil distance Ds of the image sensor14 are the same. In this case, the exit pupil 400 of the taking lens 300is divided roughly equally by the first pupil region 501 and the secondpupil region 502. Taking light receiving sensitivity distribution withrespect to the incident angle into consideration, an incident angle θ atwhich light is incident at each image height is substantially the sameas an incident angle θc at which sensitivity is the highest in pixelshaving the photodiodes 201 a and 201 b (θ=θc). In other words, in thelight receiving sensitivity distribution with respect to the incidentangle shown in FIG. 6, in the case where the incident angle θc is set asan origin, signal intensity ratios of RGB between a pair of signals thatare used for focus detection become substantially the same at any imageheight as a result of receiving a light beam of a region symmetricalrelative to the vertical axis.

FIG. 7B is a diagram showing the case where the exit pupil distance D1of the taking lens 300 is shorter than the set pupil distance Ds of theimage sensor 14. In addition, FIG. 7C is a diagram showing the casewhere the exit pupil distance D1 of the taking lens 300 is longer thanthe set pupil distance Ds of the image sensor 14. In both cases, at aperipheral image height of the image sensor 14, the exit pupil 400 ofthe taking lens 300 and the first and second pupil regions deviate, andthe exit pupil 400 of the taking lens 300 is unequally divided.Considering the light receiving sensitivity distribution with respect tothe incident angle, the higher the image height is, the more the lightincident angle θ in FIGS. 7B and 7C deviates from the incident angle θcat which the sensitivity of pixels having the photodiodes 201 a and 201b is the highest. Therefore, as the deviation between the exit pupildistance D1 of the taking lens 300 and the set pupil distance Ds islarger, and the image height is higher, in the light receivingsensitivity distribution with respect to the incident angle shown inFIG. 6, the photodiodes 201 a and 201 b will receive a light beam at anincident angle more separated from the origin. Accordingly, thedifference in intensity of RGB signals between a pair of signals thatare used for focus detection becomes larger. On the other hand, at thecentral image height of the image sensor 14, regardless of therelationship between the exit pupil distance D1 of the taking lens 300and the set pupil distance Ds of the image sensor 14, pupil deviationdoes not occur. Therefore, the pupil is equally divided, and adifference in intensity of RGB signals between a pair of signals thatare used for focus detection does not occur. In this embodiment, asdescribed above, in the case where the focal distance of the taking lens300 reaches the wide angle end or the telephoto-end, the states in FIGS.7B and 7C can be achieved. In addition, the same applies to the casewhere the taking lens is replaced, and the like.

FIG. 8 is a diagram showing an example of focus detection regions 601and 602 that are set in a shooting range 600. The focus detection region601 is a focus detection region whose center is set at a so-calledcentral image height that matches the intersection between the opticalaxis of the taking lens 300 and the image sensor 14. On the other hand,the focus detection region 602 is a focus detection region whose centeris set at a so-called peripheral image height that is separated from theintersection between the optical axis of the taking lens 300 and theimage sensor 14. In the case of performing focus detection using outputof pixels of the image sensor 14, both in a contrast detection methodand a phase difference detection method, output of pixels included inregions in the image sensor 14 corresponding to the focus detectionregions 601 and 602 are used. Accordingly, it can also be said that thefocus detection regions 601 and 602 are set in the image sensor 14.Therefore, the focus detection regions 601 and 602 will be described aspixel regions of the image sensor 14 below for ease of description. Inaddition, the pixels 200 having the configuration shown in FIG. 2A arearranged in four rows and 2N columns in the focus detection regions 601and 602. Note that this is merely exemplary, and the number and size offocus detection regions (the number of pixels included therein) can bedetermined as appropriate in a range in which phase difference detectionis not disturbed. In addition, not only regions long in the horizontaldirection but also regions long in the vertical direction may be used asfocus detection regions.

FIG. 9 shows pixels in four rows and 2N columns arranged in the focusdetection regions 601 and 602. In this embodiment, the photodiode 201 aand output of the photodiode 201 a used for obtaining a signal of an Aimage for AF in an i-th row and a j-th column are indicated by A (i, j).Similarly, the photodiode 201 b and output of the photodiode 201 b usedfor obtaining a signal of a B image for AF in the i-th row and the j-thcolumn are indicated by B (i, j). In FIG. 9, pixels having red (R) colorfilters and pixels having green (Gr) color filters are alternatelyarranged in the first and third rows, and pixels having green (Gb) colorfilters and pixels having blue (B) color filters are alternatelyarranged in the second and fourth rows. In this embodiment, green pixelsarranged next to red (R) pixels in the right-left direction areexpressed as Gr pixels, and green pixels arranged next to blue (B)pixels in the right-left direction are expressed as Gb pixels, in orderto distinguish the green pixels.

FIG. 10A is a diagram showing, separately for each arrangement of colorfilters, an example of intensity distribution of signals obtained frompixels arranged in the focus detection region 601. The horizontal axisindicates pixel number, and the vertical axis indicates signal intensitycorresponding to light amount. Two vertically adjacent graphs are shownsuch that the relative positional relationship between an A image and aB image is easily understood. The upper graph indicates an A image, andthe lower graph indicates a B image.

In the two graphs in FIG. 10A, a red (R) signal is indicated by along-broken line, a green (Gr) signal is indicated by a short-brokenline, a green (Gb) signal is indicated by a solid line, and a blue (B)signal is indicated by an alternate long and short dash line. Note thatsubstantially the same outputs are obtained from the two green signals(Gr and Gb), and thus are shown as being overlapped on each other. Onthe other hand, deviation of the heights of the peaks of the red andblue signals from those of green signals is mainly caused by thedifference in the spectral characteristics of the subject.

In addition, FIG. 10A shows a state where, due to being nearly in focus,image deviation does not occur between the A image and the B image, andthe centroids match. As described with reference to FIGS. 7A to 7C, thefocus detection region 601 is at the central image height, and thus thedegree of vignetting due to the taking lens 300 and the like aresubstantially the same, and the difference of signals between the Aimage and the B image is small.

FIG. 10B is a diagram showing, separately for each arrangement of colorfilters, an example of intensity distribution of signals that areobtained from pixels arranged in the focus detection region 602. Thehorizontal axis indicates pixel number, and the vertical axis indicatessignal intensity corresponding to light amount. The two verticallyadjacent graphs are shown such that the relative positional relationshipbetween an A image and a B image is easily understood. The upper graphindicates an A image, and the lower graph indicates a B image.

In the two graphs in FIG. 10B, a red (R) signal is indicated by along-broken line, a green (Gr) signal is indicated by a short-brokenline, a green (Gb) signal is indicated by a solid line, and a blue (B)signal is indicated by an alternate long and short dash line. Note that,in the A image (the upper graph), substantially the same outputs areobtained from the two green signals (Gr and Gb), and thus are shown asbeing overlapped on each other. On the other hand, deviation of theheights of the peaks of the red and blue signals from that of greensignals is mainly caused by the difference in the spectralcharacteristics of the subject. In the graph of the A image, the pixelpositions of the centroids of the red and blue signals are furtherdeviated. This indicates that line images are formed at differentpositions according to colors due to the magnification chromaticaberration of the taking lens 300.

The B image (the lower graph) in FIG. 10B indicates that outputsdifferent both in height of peak and pixel position of the centroid areobtained from the two green signals (Gr and Gb). The cause of signaloutputs of Gr and Gb having different shapes despite the green signalshaving the same spectral characteristics will be described later. On theother hand, deviation of the heights of the peaks of the red and bluesignals from that of the green signals is mainly caused by difference inthe spectral characteristics of the subject. In the graph of the Bimage, the positions of the red and blue signal in the horizontal axialdirection are further deviated. This indicates that line images areformed at different positions according to colors due to magnificationchromatic aberration of the taking lens 300.

In addition, in FIG. 10B, regarding any colors, an output of a signal ofthe B image is smaller than an output of a signal of the A image. Thisindicates that asymmetrical vignetting has occurred in a light beam ofthe A image and the B image passing through the taking lens 300. Asdescribed with reference to FIG. 7B, due to difference in lightreceiving sensitivity distribution with respect to the incident angle ofthe A image signals and the B image signals corresponding to theincident angle of a shooting light beam, output of signals of the Bimage is smaller than output of signals of the A image.

Furthermore, in FIG. 10B, intensity ratios, with respect to colors(RGB), of the A image and the B image differ. The peaks of the signalsof the A image is smaller in the order of G, R, and B, while the peaksof the signals of the B image is smaller in the order of R, G, and B.This indicates that the sensitivity distribution of a light beam that isincident to pixels at a peripheral image height for each color differsbetween the A image and B image. The intensity ratios of the A image andB image with respect to the colors (RGB) differ due to difference of RGBratios of light receiving sensitivity distribution corresponding to theincident angle of the A image signal and B image signal, as describedwith reference to FIG. 7B.

Next, the cause of the difference of the shapes of outputs of the twogreen signals (Gr and Gb) of the B image in FIG. 10B will be describedwith reference to FIGS. 11A and 11B. In this embodiment, difference inoutput between Gr and Gb of the B image is caused by the influence ofthe signal intensity ratios of the A image and B image, themagnification chromatic aberration of the taking lens 300, anisotropy ofcrosstalk between the pixels, and the like. Detailed description will begiven below.

FIG. 11A is a diagram showing paths along which crosstalk travels to Grpixels. FIG. 11B is a diagram showing paths along which crosstalktravels to Gb pixels. It is essentially necessary to consider crosstalkbetween pixels bi-directionally, but the amount of crosstalk thatinvades Gr and Gb pixels will be described below focused on thedifference between Gr and Gb. In addition, crosstalk between an A imageand a B image within the same pixel acts so as to cancel the imagedeviation amount, and causes a focus detection error after all. However,the light amount ratios of the A image and B image are substantially thesame regarding both Gr and Gb pixels, and thus the ratio of crosstalkamount is also the same, which does not cause a difference between Grand Gb of the B image. Therefore, a description thereof is omitted.

First, Gr and Gb of an A image will be described with reference to FIGS.11A and 11B. Gr of the A image receives crosstalk (CT_h1) from R of theB image on the left. Note that the light amount of R of the B image issmaller than Gr of the A image as shown in FIG. 10B, and thus outputchange due to crosstalk is small. Furthermore, Gr of the A imagereceives crosstalk (CT_v1) of B of the A image from above and below.Note that the light amount of B of the A image is smaller than that ofGr of the A image as shown in FIG. 10B, and thus output change due tocrosstalk is small. Accordingly, Gr of the A image is hardly affected bycrosstalk under the condition as shown in FIG. 10B.

Gb of the A image receives crosstalk (CT_h3) from B of the B image onthe left as shown in FIG. 11B. Note that the light amount of B of the Bimage is smaller than Gb of the A image as shown in FIG. 10B, and thusoutput change due to crosstalk is small. Furthermore, Gb of the A imagereceives crosstalk (CT_v3) from R of the A image from above and below.The light amount of R of the A image is relatively larger than that ofGb of the A image as shown in FIG. 10B. Note that, in the pixels of theimage sensor 14 of this embodiment, the amounts of crosstalk that occursin the horizontal and vertical directions are different, and thecrosstalk amount in the vertical direction is smaller. Therefore. Gb ofthe A image is hardly affected by crosstalk under the condition as shownin FIG. 10B. As a result, Gr and Gb of the A image do not differlargely.

Next, Gr and Gb of the B image will be described with reference to FIGS.11A and 11B. Gr of the B image receives crosstalk (CT_h2) from R of theA image on the right. The light amount of R of the A image is largerthan that of Gr of the B image as shown in FIG. 10B, and output changedue to crosstalk is large. Therefore, output of Gr of the B imagechanges due to crosstalk at the pixel position (the horizontal axis) ofR of the A image in FIG. 10B. Furthermore, Gr of the B image receivescrosstalk (CT_v2) of B of the B image from above and below. Note thatthe light amount of B of the B image is smaller than that of Gr of the Bimage as shown in FIG. 10B, and thus output change due to crosstalk issmall. Additionally, under the condition as shown in FIG. 10B, Gr of theB image is mainly affected by R of the A image, and the position of thecentroid changes due to output change.

As shown in FIG. 11B, Gb of the B image receives crosstalk (CT_h4) fromB of the A image on the right. Note that, as shown in FIG. 10B, thelight amount of B of the A image is smaller than that of Gb of the Bimage, and thus output change due to crosstalk is small. Furthermore, Gbof the B image receives crosstalk (CT_v4) of R of the B image from aboveand below. The light amount of R of the B image is relatively largerthan that of Gb of the B image as shown in FIG. 10B. Note that, in thepixels of the image sensor 14 of this embodiment, amounts of crosstalkthat occurs in the horizontal and vertical directions are different, andthe amount of crosstalk that occurs in vertical direction is smaller.Therefore, Gb of the B image is hardly affected by crosstalk under thecondition as shown in FIG. 10B. As a result, regarding Gr and Gb of theB image, output of Gr of the B image is larger than that of Gb mainlydue to crosstalk from R of the A image, and the position of the centroidchanges and deviates to the left in FIG. 10B.

Due to such effect of crosstalk, despite being not defocused, theposition of the centroid of the line spread function of Gr of the Bimage is deviated to the left in FIG. 10B relative to Gb of the B image.Focus detection is equivalent to calculation of an image deviationamount (centroid interval) between the A image and the B image, and thusa centroid position error of Gr is a focus detection error. In view ofthis, in this embodiment, by performing weighting for each color whenperforming combining of signals, a signal that can cause a focusdetection error is excluded, and accurate focus detection is realized.

A signal combining method will be described in detail later, but theoverview thereof will be described here.

As described above, the effect of crosstalk is large in the case wherethe crosstalk amount is large and the signal amount of pixels thatreceived crosstalk is small. CT_h2/B-Gr that is a ratio of the amount ofcrosstalk (CT_h2) that leaked from R pixels of the A image into Grpixels of the B image as shown in FIG. 11A to the signals of Gr pixelsof the B image (B-Gr) is referred to as a first ratio. In addition,CT_h4/B-Gb that is a ratio of the amount of crosstalk (CT_h4) thatleaked from B pixels of the A image to Gb pixels of the B image as shownin FIG. 11B to the signals of Gb pixels of the B image (B-Gb) isreferred to as a second ratio. The larger each of these ratios is, thelarger the change in position of the centroid of pixels corresponding tothe denominator of the ratio is, due to output change caused bycrosstalk. In the case where the first ratio is larger than the secondratio, the effect of crosstalk is larger in Gr pixels of the B imagethan in Gb pixels of the B image. The position of the centroid of Gbpixels of the B image is closer to the essential position of thecentroid without crosstalk. Therefore, in this embodiment, focusdetection calculation is performed with a larger weighting for signalsof Gb pixels of the B image than signals of Gr pixels of the B image.Accordingly, it is possible to reduce a change in the position of thecentroid caused by crosstalk, and accurately calculate an imagedeviation amount (centroid interval) between the A image and the Bimage. Note that, in this embodiment, description was given focused onGr pixels and Gb pixels of the B image, but the same applies to otherdifferent pixels. Note that signal (B-Gr), signal (B-Gb), and the likeare respectively output by photodiodes (photoelectric conversionportions) of Gr, photodiodes (photoelectric conversion portions) of Gb,and the like, but these photodiodes hereinafter referred to as “pixels”for convenience of description.

As seen from the above description, the followings are conceivable ascauses of different outputs being obtained from pixels Gr and Gb fromwhich the same outputs are essentially obtained:

-   spectral characteristics of a subject (RGB ratio):-   angle dependence and spectral dependence of sensitivity distribution    of a pair of photodiodes with respect to the incident angle:-   the taking lens and the image height of the focus detection region;-   magnification chromatic aberration that occurs in the taking lens;    and-   crosstalk amount between pixels (for each direction in which    crosstalk travels from adjacent pixels).

Different outputs are obtained from Gr and Gb pixels due to thesefactors, and thus it is difficult to obtain the degree of contributionof each factor, and obtain the degree of influence of a detection errorfor an obtained focus detection result for each factor. In addition, thespectral characteristics of the subject also contributes to thedifference, and thus it is also difficult for the image capturingapparatus to store crosstalk amounts and focus detection errors ascorrection values in advance.

In view of this, in this embodiment, the reliability of a focusdetection result is determined using the difference in output between Grand Gb that occurs as a result. Accordingly, even in the case where acrosstalk amount different from a design value and a focus detectionerror occur due to a manufacturing error and the like, it is possible toperform accurate reliability determination. A reliability determinationmethod will be described later in detail, but, in this embodiment,reliability determination to be described later is performed using Gb ofthe B image that receives light whose spectrum is substantially the sameas Gr of the B image that receives green light.

Focus Detection Operation

Next, a focus adjustment operation of the camera 100 will be describedwith reference to a flowchart shown in FIG. 12. Note that processingshown in FIG. 12 is carried out in the state where the main mirror 130and the sub mirror 131 retreated to the outside of the light path(mirror lock-up), more specifically, during live view display (whenshooting a moving image to be displayed) or during recording of a movingimage (when shooting a moving image to be recorded). Note that, here,description is given in which automatic focus detection of a phasedifference detection method is performed using output of the imagesensor 14, but, as described above, automatic focus detection of acontrast detection method can also be performed.

In step S701, the system control unit 50 determines, based on anoperation performed on the switch SW1 (62), the operation unit 70, orthe like, whether or not a focus detection start instruction has beeninput, and in the case where it is determined that a focus detectionstart instruction has been input, advances the procedure to step S702,and in the case where it is not determined that a focus detection startinstruction has been input, waits. Note that the system control unit 50may advance the procedure to step S702 using start of live view displayor moving image recording as a trigger, regardless of input of a focusdetection start instruction.

In step S702, the system control unit 50 acquires lens frame informationof the taking lens 300 and various pieces of lens information such as afocus lens position from the lens system control unit 346 via theinterface units 38 and 338 and the connectors 122 and 322.

In step S703, the system control unit 50 instructs the image processingunit 20 to connect signals of photoelectric conversion portions in thepupil divided direction, the signals having been obtained from pixeldata in a focus detection region of frame image data that is beingsequentially read out, and generate pairs of image signals for AF. Theimage processing unit 20 generates pairs of image signals for AF, andsupplies the generated pairs of image signals for AF to the AF unit 42.The AF unit 42 performs processing for correcting the difference insignal level and the like on the pairs of image signals for AF. Also,the AF unit 42 detects a peak value (maximum value) and a bottom value(minimum value) of the image signals for AF. The image processing unit20 performs processing for each type of the color filters (R, Gr, Gb,and B) of pixels, and generates pairs of image signals, as pairs ofimage signals for AF.

In step S704, the AF unit 42 performs reliability determinationprocessing using Gr and Gb signals of an A image or a B image. Asdescribed above, reliability in the case where crosstalk that occurreddue to various reasons affects focus detection is determined using afact that the same signal wave forms are obtained from the Gr and Gbsignals of the A image or the B image in the case where there is nocrosstalk. This processing will be described later in detail. The AFunit 42 outputs the reliability determination result to the systemcontrol unit 50.

In step S705, the system control unit 50 determines the reliability offocus detection signals based on the reliability determination resultobtained from the AF unit 42 in step S704. In the case where it isdetermined that the reliability is high, the procedure advances to stepS706, and the image processing unit 20 processes pairs of image signalsfor AF configured according to each type of the color filters (R, Gr,Gb, and B) of pixels into Y signals (luminance signals) that do notchange by color, by performing filter processing and combing processing.Accordingly, it is possible to reduce the information amount of thepairs of image signals for AF, and reduce the calculation amount insubsequent processing. In this embodiment, pixels in four rows and 2Ncolumns are arranged in the focus detection regions 601 and 602, but,for example, two pixels are combined in each of the horizontal andvertical directions, and the pixels are compressed into signals in tworows and N columns.

On the other hand, in the case where it is determined in step S705 thatthe reliability of the focus detection signals is low, the procedureadvances to step S707. By performing filter processing and combiningprocessing after performing weighting processing for each color, theimage processing unit 20 processes pairs of image signals for AFconfigured according to each type of the color filters (R, Gr, Gb, andB) of pixels into signals (combined signals) that do not change bycolor. For example, in the configuration of this embodiment, Gr of the Bimage is likely to be affected by crosstalk, and thus it is conceivablethat weighting is performed such that the ratio of Gr signals to focusdetection signals is decreased, and signals for each type of the colorfilters (R, Gr, Gb, and B) are combined. Weighting includes generationof focus detection signals without using Gr (setting weight of Gr tozero). In this embodiment, pixels in four rows and 2N columns arearranged in the focus detection regions 601 and 602, but the pixels arecompressed into signals in two rows and N columns similar to step S705while performing weighting for each color.

Weighting for each color changes according to the following pieces ofinformation:

-   light receiving sensitivity distribution for the absolute value    difference between the exit pupil distance of a taking lens and the    set pupil distance of an image sensor, and the incident angle of the    image sensor;-   magnification chromatic aberration of a taking lens;-   spectral characteristics of a subject (RGB ratio); and-   structure of an image sensor (whether or not there is a light    shielding wall, the structure between pixels or the structure of    interconnect layers, and the like).

The above-described pieces of information will be described individuallybelow.

Light Receiving Sensitivity Distribution for Absolute Value DifferenceBetween Exit Pupil Distance of Imaging Optical System and Set PupilDistance of Image Sensor, and Incident Angle of Image Sensor

As described with reference to FIGS. 7A to 7C, as the absolute value ofthe difference between the exit pupil distance of the taking lens 300and the set pupil distance of the image sensor 14 is larger, and theimage height is higher, the photodiodes 201 a and 201 b receive a lightbeam at an incident angle separated from the origin, in the lightreceiving sensitivity distribution with respect to the incident angleshown in FIG. 6. Therefore, the difference in RGB signal intensity inpairs of signals that are used for focus detection is large.Accordingly, different outputs are likely to be generated from theabove-described Gr and Gb pixels. Therefore, it is possible to reducethe effect of the crosstalk amount, and reduce focus detection errors,by increasing weighting of pixels (e.g., Gb) in which the degree of theeffect of crosstalk is small as the absolute value difference betweenthe exit pupil distance of a taking lens and the set pupil distance ofan image sensor is larger, or the image height is higher.

Magnification Chromatic Aberration of Imaging Optical System

As described with reference to FIGS. 10A and 10B, if the pixel positionof the centroid of red (R), green (G), and blue (B) signals is deviateddue to the magnification chromatic aberration of the taking lens 300,line images are formed at positions different according to colors.Therefore, for example, it is preferable that the camera 100 or thetaking lens 300 holds information regarding the centroid differencebetween R and G and the centroid difference between B and G as themagnification chromatic aberration information of the taking lens 300,and weighting for each color is changed according to the amount ofcentroid difference of line images between R and G and the amount ofcentroid difference of line images between B and G. For example, in thecase where the signal amount of G is large, the centroid differencebetween R and G is large, and the centroid difference between B and G islarge, it is possible to reduce the crosstalk amount, and reduce focusdetection errors by increasing weighting of Gr and B.

Spectral Characteristics of Subject (RGB Ratio)

The degree of the effect of crosstalk differs according to the spectralcharacteristics of a subject. For example, in the case of a subjecthaving only green spectral characteristics, Gr (Gb) pixels receive thelargest amount of subject light, and signal intensity of the otherpixels receive crosstalk from Gr (Gb), and thus it is preferable thatthe other pixels are not used for signals for focus detection. Bychanging weighting for each color according to the intensity at whichsubject light is received based on the spectral characteristics of thesubject in this manner, it is possible to reduce the crosstalk amount,and reduce focus detection errors. Regarding the spectralcharacteristics of the subject, the RGB ratio of a focus detectionregion is calculated by the image processing unit 20 in advance, andaccording to the calculation result, weighting of signals for focusdetection for each color is performed. In addition, weighting may bechanged according to of the brightness of the subject.

Structure of Image Sensor

As described above, anisotropy of interconnect layers of a pixel and theheight of barriers against signal charges between pixels differaccording to the pixel pitch and the pixel structure of the imagesensor. The amount of crosstalk that occurs differs due to thesestructures, and thus in the case of a structure according to whichcrosstalk is likely to occur (e.g., barriers against signal chargesbetween pixels are low), weighting for each of the colors (R, Gr, Gb,and B) is performed (weight of pixels in which the effect of crosstalkis small is increased). In addition, in the case of a structureaccording to which crosstalk is unlikely to occur (e.g., barriersagainst signal charges between pixels are large), weighting for eachcolor is reduced. These make it possible to reduce focus detectionerrors.

In addition, weighting is not limited to Gr and Gb, and may be performedon other colors in addition to Gr and Gb. It suffices that weighting isperformed on red (R) and blue (B) outputs according to the amount ofmagnification chromatic aberration and the like, which have beendescribed above. For example, in the case where deviation of the imageforming position between R and G is large due to magnification chromaticaberration, it is sufficient that, in step S707 in FIG. 12, weighting ofred is made small in the processing performed in step S706.

Returning to FIG. 12, in step S708, the AF unit 42 calculates acorrelation amount COR (k) of the A image and the B image, and detectsthe deviation amount k at which the correlation amount COR (k) isminimized, as the deviation amount (phase difference) of the image.Furthermore, the AF unit 42 converts the detected deviation amount intoa defocus amount. Detailed description of this processing will be givenlater. The AF unit 42 outputs the defocus amount to the system controlunit 50.

In step S709, the system control unit 50 determines a focus lens drivingamount and driving direction of the taking lens 300 based on the defocusamount obtained from the AF unit 42 in step S708.

In step S710, the system control unit 50 transmits information regardingthe driving amount and the driving direction of the focus lens to thelens system control unit 346 of the taking lens 300 via the interfaceunits 38 and 338 and the connectors 122 and 322. The lens system controlunit 346 transmits the information regarding the driving amount and thedriving direction of the focus lens to the focus control unit 342. Thefocus control unit 342 drives the focus lens based on the receivedinformation regarding the lens driving amount and driving direction.Accordingly, focus adjustment of the taking lens 300 is performed. Notethat the operation in FIG. 12 may be continuously carried out also whenmoving image data of the next frame onward is read out. The informationregarding the driving amount and the driving direction of the focus lensmay be directly transmitted from the system control unit 50 to the focuscontrol unit 342.

Subroutine of Reliability Determination Using G Pixels

Next, a subroutine of reliability determination using G pixels that isperformed in step S704 in FIG. 12 will be described with reference tothe flowchart in FIG. 13. In step S7041, the AF unit 42 selects signalsfor which reliability determination is performed. As described above,the effect of crosstalk is larger on signals whose light amount is smalldue to vignetting, and thus signals whose light amount is small areselected. In the situation described with reference to FIG. 10B, the Bimage is selected.

In step S7041 onward, the AF unit 42 compares signals of Gr of the Bimage and Gb of the B image. Contents of subsequent comparisonprocessing will be described with reference to FIG. 14. In FIG. 14,signals of Gr and Gb of the B image in FIG. 10B are extracted andenlarged in the vertical axial direction for ease of understanding.

In step S7042, the AF unit 42 compares the difference between a peak anda bottom in two signals. As shown in FIG. 14, P-B_Gr and P-B_Gb that arepeaks of the two signals are compared. In the case where there is adifference that is larger than or equal to a predetermined threshold,the effect of crosstalk is large, and it is determined that thereliability of focus detection using all of the signals is low. Focusdetection using all of the signals refers to focus detection using Ysignals that is performed in step S706. In this embodiment, in the casewhere it is determined that the reliability of focus detection using Ysignals is low, focus detection using signals that underwent weightingprocessing for each color as described above in step S707 is performed.

In step S7043, the AF unit 42 compares contrast information of Gr andGb. It suffices that the absolute value sum of the difference betweenadjacent signals, the square sum of the difference between adjacentsignals, or the like is used as contrast information. Accordingly, theintensity difference of amplitude of signals can be detected. In stepS7043, in the case where it is found that there is a difference incontrast information of Gr and Gb that is larger than or equal to apredetermined threshold, it is concerned that the effect of crosstalk islarge, and thus it is determined that the reliability of focus detectionusing all of the signals is low.

In step S7044, the AF unit 42 detects an image deviation amount betweenGr and Gb of the B image. An image deviation amount detecting methodthat is performed during focus detection to be described later is usedfor calculating an image deviation amount. Determination in steps S7042and S7043 is reliability determination that is focused on the lightamount (amplitude) of crosstalk, but, in step S7044, deviation(difference in phase) of the position of the centroid of signals due tocrosstalk is detected, and the reliability is determined. A method forcalculating an image deviation amount will be described in the followingdescription of a focus detection method. In step S7044, in the casewhere the image deviation amount between Gr and Gb of the B image islarger than or equal to a predetermined threshold, it is determined thatthe reliability of focus detection using all of the signals is low.

In step S7045, the AF unit 42 acquires subject spectrum information asoutput of the photometry unit 46, and estimate the effect of crosstalkdue to subject spectrum. As described above, a focus detection error dueto crosstalk is under various effects of the spectral characteristics.On the other hand, the closer the spectrum of a light beam received fromthe subject is to a single wavelength, the smaller the effects become.This is because magnification chromatic aberration does not occur in asingle wavelength, sensitivity distribution of photodiodes with respectto the incident angle does not depend on the spectrum, and pixels fromwhich outputs are obtained are restricted after a light beam passedthrough the color filters. In step S7045, it is determined whether ornot the subject spectrum is close to the single wavelength, and thedegree of the effect of crosstalk is determined.

In addition, in step S7045, also in the case where the subject spectrumis not a single wavelength, the degree of the effect of crosstalk isfurther determined. In steps S7042 to S7044, the effect of crosstalk isdetermined using Gr and Gb of the B image. However, in the case wherethe spectrum of the subject includes a large amount of red and bluecomponents and a small amount of green components, the influence on thefocus detection result is small even if there is a difference between Grand Gb of the B image. In view of this, in step S7045, the rate of greenoutput to the entire light amount is calculated using output of thephotometry unit 46, and in the case where the ratio of green is largerthan or equal to a predetermined threshold, it is determined that theeffect of crosstalk is large, and it is determined that the reliabilityof focus detection using all of the signals is low.

In step S7046, the AF unit 42 comprehensively determines the reliabilityof focus detection using all of the signals, based on the result ofreliability determination performed in steps S7042 to S7045. Thereliability of the determination results in steps S7042 to S7044 isdetermined to be high, only in the case where it is determined that thereliability is high in all of the signals. If the degree of the effectis considered to be small also in the case where it is determined insteps S7042 to S7044 that the reliability is low using the resultobtained in steps S7045, the reliability of focus detection using all ofthe signals is determined to be high.

When step S7046 is complete, the AF unit 42 ends the subroutine ofreliability determination using G pixels, and advances the procedure tostep S705 in FIG. 12.

In this embodiment, as described above, the amount of crosstalk differsaccording to the following factors:

-   the spectral characteristics of the subject (RGB ratio);-   angle dependence and spectral dependence of sensitivity distribution    of a pair of photodiodes with respect to the incident angle:-   the imaging optical system and the image height of the focus    detection region;-   magnification chromatic aberration that occurs in the imaging    optical system; and-   the crosstalk amount between pixels (for each direction in which    crosstalk travels from adjacent pixels).

Therefore, the amount of errors that occur during focus detection isaffected by the degree of above-described factors. Therefore, it issufficient that thresholds of determinations used during reliabilitydetermination using G pixels are changed in light of the above-describedfactors. For example, it suffices for the thresholds to be set such thatreliability determination can be more strictly performed in the case ofa higher image height, a larger magnification chromatic aberrationamount, and a larger crosstalk amount. Note that, in this embodiment, anexample has been described in which two photoelectric conversionportions are provided in one pixel in the horizontal direction, but thecrosstalk amount between pixels differs also according to thearrangement direction and the number of photoelectric conversionportions. Therefore, reliability determination may be performed takingthese factors into consideration in order to perform more accuratedetermination.

Next, a subroutine of processing for calculating a defocus amount thatis performed by the AF unit 42 in step S708 in FIG. 12 will be furtherdescribed with reference to the flowchart shown in FIG. 15. In stepS7081, the AF unit 42 selects a row that is a calculation target from aplurality of rows in a focus detection region, and performs correlationcalculation. In this embodiment, the first row of the signals of tworows and N columns that underwent signal generation in steps S706 andS707 is selected.

In focus detection of a phase difference detection method, a pair ofimages having a portion corresponding to the same subject are generated,a phase difference (deviation amount) of the pair of images is detected,and the phase difference is converted into a defocus amount and adefocus direction. A sequence of signals (an A image) that are based onsignals obtained from the photodiodes 201 a of a plurality of pixels 200that are in a predetermined direction (e.g., the horizontal direction)and a sequence of signals (a B image) that are based on signals obtainedfrom the photodiodes 201 b are equivalent to images of the same subjectviewed from different viewpoints. Therefore, by detecting the phasedifference between the A image and the B image, and converting the phasedifference into a defocus amount and a defocus direction, focusdetection of a phase difference detection method can be realized.

It is then possible to calculate a value (correlation amount) indicatingthe correlation between the A image and the B image at individualpositions while changing the relative distance (shift amount) betweenthe A image and the B image in a predetermined direction, and detect ashift amount at which the correlation is the highest as a phasedifference between the A image and the B image. For example, thecorrelation amount may be a difference accumulation value ofcorresponding signal values, or may be another value.

For example, assuming that A (1, 1) to A (1, N) are generated as an Aimage, and B (1, 1) to B (1, N) are generated as a B image, and theshift amount k is changed in units of pixels in the range of−kmax≤k≤kmax, the correlation amount COR (k) at each relative positioncan be calculated as follows. Note that, here, A (M, N) and B (M, N)respectively indicate a signal of the A pixel and a signal of the Bpixel in an M-th row and an N-th column.

$\begin{matrix}{{{{COR}(k)} = {\sum\limits_{i = 1}^{N - 1 - {2 \times k\;\max}}{{{A\left( {i - k} \right)} - {B\left( {i + k} \right)}}}}}\left( {{{- k}\;\max} \leq k \leq {k\;\max}} \right)} & (1)\end{matrix}$In step S7081, a correlation amount COR is calculated from the signalsof the A image and the B image of the selected row.

In step S7082, determination regarding row addition of correlationamounts is performed. There are a plurality of rows in a focus detectionregion, and in the case of performing correlation calculation, additionof correlation amounts is performed. Note that, there are some rows fromwhich a reliable focus detection result is not obtained due tosaturation, from among rows in which correlation calculation isperformed. Therefore, determination is made regarding whether or not toadd the correlation amount obtained in step S7082 to a correlationamount added in advance. In the case where the reliability of thecorrelation amount calculated in step S7081 is high, it is determined toperform addition.

In step S7083, in the case of performing addition based on the result ofdetermination performed in step S7082, the procedure advances to stepS7084, and the correlation amount obtained in step S7081 is added to theaddition result of correlation amounts obtained in advance. On the otherhand, if it is determined in step S7083 not to perform addition, stepS7084 is skipped.

Next, in step S7085, it is determined whether or not correlationcalculation has been performed on all of the rows. If correlationcalculation has not been performed on all of the rows, the procedurereturns to step S7081, and the procedure continues. If correlationcalculation has been performed on all of the rows in the focus detectionregion, the procedure advances to step S7086, where a defocus amount iscalculated. First, the value of the shift amount k at which COR (k)obtained after addition is minimized is obtained. Here, the shift amountk calculated in Expression 1 is an integer, but the shift amount k thatis lastly obtained is a real number in order to improve the resolution.For example, in the case where a minimum value that is obtained inExpression 1 is COR(a), based on interpolation calculation using COR(a−1), COR (a), and COR (a+1), and the like, a shift amount that is areal number value, and at which the correlation amount in this sectionis minimized is obtained.

In this embodiment, a shift amount dk at which the sign of thedifference value of the correlation amount COR changes is calculated asthe shift amount k at which a correlation amount COR1 (k) is minimized.

First, the AF unit 42 calculates a difference value DCOR of thecorrelation amount in accordance with Expression 2 below.DCOR(k)=COR1(k)−COR1(k−1)  (2)

The AF unit 42 then obtains the shift amount dk at which the sign ofdifference amount changes using the difference value DCOR of thecorrelation amount. Letting that the value of k immediately before thesign of the difference amount changes be k1, and the value of k at whichthe sign changed be k2 (k2=k1+1), the AF unit 42 calculates the shiftamount dk in accordance with Expression 3 below.dk=k1+|DCOR(k1)|/|DCOR(k1)−DCOR(k2)|  (3)

In this manner, the AF unit 42 calculates the shift amount dk at whichthe correlation amount between an A image and a B image is the largestin units of sub pixels. Note that a method for calculating a phasedifference of two one-dimensional image signals is not limited to thosedescribed here, and any known method can be used.

Subsequently, the shift amount dk obtained in step S7086 is multipliedby a predetermined defocus conversion coefficient, and is converted intoa defocus amount Def. Here, the defocus conversion coefficient can beobtained based on optical conditions (e.g., aperture, exit pupildistance, and lens frame information) during shooting, the image heightof a focus detection region, the sampling pitch of signals constitutingan A image and a B image, and the like.

When calculation of a defocus amount is complete in step S7086, asubroutine of processing for calculating a defocus amount ends, and theprocedure advances to step S709 in FIG. 12.

As described above, according to this embodiment, reliabilitydetermination is performed by detecting the difference of featureamounts of signals, using two signals (Gr and Gb) from which the sameoutput is essentially envisioned to be obtained by sampling a subjectimage passing through a taking lens. With such a configuration, even inthe case where signals include an error due to the effect of crosstalk,crosstalk correction, and the like, a highly reliable focus detectionresult can be obtained in terms of detection accuracy.

Furthermore, selection of signals that are used when performing focusdetection processing is realized through weighting addition for eachcolor during signal compression, based on the reliability determinationresult. Accordingly, only in the case where there is an effect ofcrosstalk, affected signals are excluded, and an accurate focusdetection result can be obtained. In addition, in the case where thereis no effect of crosstalk, it is possible to obtain a focus detectionresult with a favorable SN ratio by using all of the signals.

In this embodiment, weighting addition for each color during signalcompression is performed based on a reliability determination result,but a method for obtaining a focus detection result from which theeffect of crosstalk is excluded is not limited thereto. For example,after focus detection results are calculated for the colors (Gr, Gb, Rand B), the focus detection results may be weighted and averaged basedon the reliability determination result.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2017-169605, filed Sep. 4 2017, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus comprising: an imagesensor that has a plurality of two-dimensionally arrayed pixels, each ofthe pixels having a first photoelectric conversion portion that receivesa light beam passing through a first pupil region of an exit pupil of animaging optical system and a second photoelectric conversion portionthat receives a light beam passing through a second pupil region of theexit pupil of the imaging optical system different from the first pupilregion; a generation unit configured to generate a first image signal,in a pupil divided direction, based on a first signal obtained bycombining a signal of the first photoelectric conversion portion to asignal of another neighboring first photoelectric conversion portion,and generate a second image signal, in the pupil divided direction,based on a second signal obtained by combining a signal of the secondphotoelectric conversion portion to a signal of another neighboringsecond photoelectric conversion portion; and a focus detection unitconfigured to detect a phase difference between the first image signaland the second image signal, wherein, in a case of combining a signal ofthe first photoelectric conversion portion to a signal of anotherneighboring first photoelectric conversion portion, or combining asignal of the second photoelectric conversion portion to a signal ofanother neighboring second photoelectric conversion portion, thegeneration unit decreases weighting of a signal of a photoelectricconversion portion in which an effect of crosstalk from a neighboringphotoelectric conversion portion is large, and performs combining. 2.The image capturing apparatus according to claim 1, further comprising:a determination unit configured to determine a reliability of adetection result of the focus detection unit, wherein, in a case wherethe determination unit determines that a reliability is low, in a caseof combining a signal of the first photoelectric conversion portion to asignal of another neighboring first photoelectric conversion portion, orcombining a signal of the second photoelectric conversion portion to asignal of another neighboring second photoelectric conversion portion,the generation unit decreases weighting of a signal of a photoelectricconversion portion in which an effect of crosstalk from a neighboringphotoelectric conversion portion is large, and performs combining. 3.The image capturing apparatus according to claim 1, further comprising:a determination unit configured to determine a reliability of adetection result of the focus detection unit wherein, in a case wherethe determination unit determines that a reliability is high, in a caseof combining a signal of the first photoelectric conversion portion to asignal of another neighboring first photoelectric conversion portion, orcombining a signal of the second photoelectric conversion portion to asignal of another neighboring second photoelectric conversion portion,the generation unit performs combining without weighting a signal ofanother neighboring photoelectric conversion portion.
 4. The imagecapturing apparatus according to claim 3, wherein each of the pluralityof pixels has one of red, green, and blue color filters, and thegeneration unit generates the first or second combined signal that is aluminance signal, by combining signals of the first or secondphotoelectric conversion portions respectively corresponding to colorfilters of different colors.
 5. The image capturing apparatus accordingto claim 2, wherein each of the plurality of pixels has one of red,green, and blue color filters, and the determination unit determines thereliability using a signal of a photoelectric conversion portioncorresponding to a green color filter.
 6. The image capturing apparatusaccording to claim 5, wherein, regarding the color filters, a row inwhich red and green are alternately arranged and a row in which greenand blue are alternately arranged are alternately arrayed, and thedetermination unit determines the reliability by comparing signals ofphotoelectric conversion portions corresponding to green color filtersof the row in which red and green are alternately arranged and signalsof photoelectric conversion portions corresponding to green colorfilters of the row in which green and blue are alternately arranged. 7.The image capturing apparatus according to claim 5, wherein thedetermination unit determines the reliability based on a spectralcharacteristics of a subject.
 8. The image capturing apparatus accordingto claim 1, wherein the generation unit changes weighting of a signal ofa photoelectric conversion portion in which an effect of crosstalk fromthe neighboring photoelectric conversion portion is large, according toa difference between an exit pupil distance of the imaging opticalsystem and a set pupil distance of the image sensor.
 9. The imagecapturing apparatus according to claim 1, wherein the generation unitchanges weighting of a signal of a photoelectric conversion portion inwhich an effect of crosstalk from the neighboring photoelectricconversion portion is large, according to an image height of the pixelthat is a distance from a center of the image sensor.
 10. The imagecapturing apparatus according to claim 1, wherein the generation unitchanges weighting of a signal of a photoelectric conversion portion inwhich an effect of crosstalk from the neighboring photoelectricconversion portion is large, according to information regardingaberration of the imaging optical system.
 11. The image capturingapparatus according to claim 1, wherein the generation unit changesweighting of a signal of a photoelectric conversion portion in which aneffect of crosstalk from the neighboring photoelectric conversionportion is large, according to spectral information of signals of afocus detection region.
 12. A controlling method of an image capturingapparatus including an image sensor that has a plurality oftwo-dimensionally arrayed pixels, each of the pixels having a firstphotoelectric conversion portion that receives a light beam passingthrough a first pupil region of an exit pupil of an imaging opticalsystem and a second photoelectric conversion portion that receives alight beam passing through a second pupil region of the exit pupil ofthe imaging optical system different from the first pupil region, themethod comprising: generating a first image signal by connecting, in apupil divided direction, based on a first combined signal obtained bycombining a signal of the first photoelectric conversion portion to asignal of another neighboring first photoelectric conversion portion,and generating a second image signal, in the pupil divided direction,based on a second combined signal obtained by combining a signal of thesecond photoelectric conversion portion to a signal of anotherneighboring second photoelectric conversion portion; and detecting aphase difference between the first image signal and the second imagesignal, wherein, in the generating, in a case of combining a signal ofthe first photoelectric conversion portion to a signal of anotherneighboring first photoelectric conversion portion, or combining asignal of the second photoelectric conversion portion to a signal ofanother neighboring second photoelectric conversion portion, weightingof a signal of a photoelectric conversion portion in which an effect ofcrosstalk from a neighboring photoelectric conversion portion is largeis decreased, and combining is performed.
 13. The method according toclaim 12, further comprising: determining a reliability of a detectionresult of the focus detecting, wherein, in a case where it is determinedthat a reliability is low, in a case of combining a signal of the firstphotoelectric conversion portion to a signal of another neighboringfirst photoelectric conversion portion, or combining a signal of thesecond photoelectric conversion portion to a signal of anotherneighboring second photoelectric conversion portion, weighting of asignal of a photoelectric conversion portion in which an effect ofcrosstalk from a neighboring photoelectric conversion portion is largeis decreased, and combining is performed.
 14. The method according toclaim 12, further comprising: determining a reliability of a detectionresult of the focus detecting wherein, in a case where it is determinedthat a reliability is high, in a case of combining a signal of the firstphotoelectric conversion portion to a signal of another neighboringfirst photoelectric conversion portion, or combining a signal of thesecond photoelectric conversion portion to a signal of anotherneighboring second photoelectric conversion portion, combining isperformed without weighting a signal of another neighboringphotoelectric conversion portion.
 15. The method according to claim 14,wherein each of the plurality of pixels has one of red, green, and bluecolor filters, and in the generating, the first or second combinedsignal that is a luminance signal is generated by combining signals ofthe first or second photoelectric conversion portions respectivelycorresponding to color filters of different colors.
 16. The methodaccording to claim 13, wherein each of the plurality of pixels has oneof red, green, and blue color filters, and in the determining, thereliability is determined using a signal of a photoelectric conversionportion corresponding to a green color filter.
 17. The method accordingto claim 16, wherein, regarding the color filters, a row in which redand green are alternately arranged and a row in which green and blue arealternately arranged are alternately arrayed, and in the determining thereliability is determined by comparing signals of photoelectricconversion portions corresponding to green color filters of the row inwhich red and green are alternately arranged and signals ofphotoelectric conversion portions corresponding to green color filtersof the row in which green and blue are alternately arranged.
 18. Themethod according to claim 12, wherein in the generating weighting of asignal of a photoelectric conversion portion in which an effect ofcrosstalk from the neighboring photoelectric conversion portion is largeis changed according to an image height of the pixel that is a distancefrom a center of the image sensor.
 19. The method according to claim 12,wherein in the generating weighting of a signal of a photoelectricconversion portion in which an effect of crosstalk from the neighboringphotoelectric conversion portion is large is changed according tospectral information of signals of a focus detection region.
 20. Animage capturing apparatus comprising: an image sensor that has aplurality of two-dimensionally arrayed pixels, each of the pixels havinga first photoelectric conversion portion that receives a light beampassing through a first pupil region of an exit pupil of an imagingoptical system and a second photoelectric conversion portion thatreceives a light beam passing through a second pupil region of the exitpupil of the imaging optical system different from the first pupilregion; and at least one processor or circuit configured to perform theoperations of the following units: a generation unit configured togenerate a first image signal, in a pupil divided direction, based on afirst signal obtained by combining a signal of the first photoelectricconversion portion to a signal of another neighboring firstphotoelectric conversion portion, and generate a second image signal, inthe pupil divided direction, based on a second signal obtained bycombining a signal of the second photoelectric conversion portion to asignal of another neighboring second photoelectric conversion portion;and a focus detection unit configured to detect a phase differencebetween the first image signal and the second image signal, wherein, ina case of combining a signal of the first photoelectric conversionportion to a signal of another neighboring first photoelectricconversion portion, or combining a signal of the second photoelectricconversion portion to a signal of another neighboring secondphotoelectric conversion portion, the generation unit decreasesweighting of a signal of a photoelectric conversion portion in which aneffect of crosstalk from a neighboring photoelectric conversion portionis large, and performs combining.